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Abstract 

A program of research is necessary to examine the psychometric properties of instruments 
designed to measure individuals’ achievement goal orientations. Recently, research on 
achievement goal orientation has examined the stability of achievement goals to assess how 
context might influence individuals’ achievement goals. Accordingly, studies are necessary to 
establish factorial invariance across contexts. We examined the psychometric properties of the 
Achievement Goal Questionnaire (AGQ; Elliot & McGregor, 2001) across task contexts within a 
single classroom environment. We tested the factor structure by comparing five competing 
models and evaluated the invariance of the factor structure across four task contexts. Results 
revealed that the hypothesized four-factor structure was replicated, construct- and discriminant- 
related evidence of validity were supported, and both internal consistency and test-retest 
reliability estimates were satisfactory. Moreover, invariance held at all levels across the various 
contexts. 

Keywords: Achievement goal orientation, measurement, factorial invariance, psychometrics, 
reliability. 


Resume 

Un programme de recherche est necessaire afin d'examiner les proprietes psychometriques 
d'instruments concurs pour mesurer les orientations des individus envers des objectifs de reussite. 
Recemment, la recherche sur l'orientation envers des objectifs de reussite s'est interessee a la 
stability de ces objectifs afin d'evaluer comment le contexte pouvait influer les orientations 
envers des objectifs de reussite chez divers individus. C'est d'ailleurs ceci qui justifie les etudes 
permettant d'etablir l’invariance factorielle quels que soient les contextes examines. A cet effet, 
nous avons examine les proprietes psychometriques du questionnaire sur les objectifs de reussite 
(AGQ ; Elliot et McGregor, 2001) dans differents contextes d’activite au sein d’un seul meme 
environnement en salle de classe. Nous avons etudie la structure factorielle en comparant cinq 
modeles differents et avons evalue l’invariance de cette structure selon quatre contextes 
d'activites. Les resultats indiquent que la structure a quatre facteurs proposee a ete reproduite, 
que la validite discriminante et des constructs ont ete soutenues, et que les coefficients estimes de 
l'uniformite interne et de la fiabilite test-rest sont satisfaisants. D'ailleurs, l'invariance a ete 
soutenue a tous les niveaux quels que soient les contextes examines. 

Mots-cles: Les orientations envers des objectifs de reussite, la mesure, les proprietes 
psychometriques, l'invariance factorielle, la fiabilite. 
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Assessing the Psychometric Properties of the Achievement Goals Questionnaire Across 

Task Contexts 

Introduction 

Imagine a teacher assigns students a task that provides opportunities for students to work 
together, allows them to select a topic for the task, and allows them to submit the assignment for 
formative feedback prior to submission for summative assessment. Three weeks later, the same 
teacher tells students they will be given an exam graded on normative standards, and scores will 
be posted in the class. Would students’ achievement goals be similar for both tasks? 
Theoretically, the likely answer is no. As Ames (1992) proposed, the various tasks and learning 
activities that teachers set for their students can have powerful influences on how students 
engage with the tasks, the amount of effort they expend, and the strategies they choose to 
complete the tasks. Specifically, tasks can influence learners’ orientations toward differing 
achievement goals. Midgley, Kaplan, and Middleton (2001) define achievement goals as “the 
purposes for behavior that are perceived or pursued in a competence-relevant setting” (p.77). 

Initially, theorists proposed two types of achievement goals: a mastery goal and a 
performance goal (e.g., Anderman & Maehr, 1994; Dweck & Leggett, 1988; Elliot & Dweck, 
1988; Maehr & Pintrich, 1991; Meece, 1991). Today, theorists conceptualize achievement goals 
within a trichotomous or 2 x 2 framework. Within the trichotomous framework (e.g., Elliot & 
Church, 1997; Middleton & Midgley, 1997; Pintrich, 2000), three distinct achievement goal 
orientations are proposed: a mastery goal, performance-approach goal, and a performance-avoid 
goal. A mastery goal orientation (or mastery-approach orientation) describes learners who strive 
to develop competence and task mastery. Learners with a mastery goal orientation are theorized 
to believe effort and outcome co-vary. In contrast, a performance-approach goal orientation 
characterizes learners who strive to demonstrate aptitude and seek favourable judgments; 
demonstrations of competence are in comparison to others. The third goal is a performance- 
avoidance orientation, whereby learners strive to avoid appearing unable and avoid negative 
judgments. Like the performance-approach orientation, comparisons of competence are made 
with other individuals. 

The 2x2 achievement goal framework (Elliot & McGregor, 2001) divides goals into a 
mastery-performance dichotomy plus an approach-avoidance dichotomy. This adds a fourth goal 
orientation, a mastery-avoidance orientation, whereby a learner’s goal is to avoid failure rooted 
in an intrapersonal perspective (relative to oneself, like the mastery-approach orientation) rather 
than in comparison to others. For the mastery-avoid goal construct, incompetence is the focus. 
A mastery-avoid oriented learner, for example, may strive to avoid misunderstanding or failing 
to learn course material, or strive not to forget what has been learned (Elliot & McGregor, 2001). 
Conceptually, the mastery component in a mastery-avoid goal orientation emerges from optimal 
antecedents (e.g., motive dispositions, implicit theories, socialization histories) that may 
facilitate positive consequences (such as mastery-approach goals; see Elliot & McGregor, 2001, 
for a complete discussion). The avoidance component, however, is hypothesized to emerge from 
non-optimal antecedents and may result in negative consequences (such as performance- 
avoidance goals). 

Several studies have examined the consequences of endorsing various goals and how 
those goals relate to achievement (Elliot, 1999). For example, performance-approach goals 
correlated with positive factors such as absorption during task involvement (Elliot & 
Harackiewicz, 1996), high performance outcomes (Elliot & Church, 1997), task value (Wolters, 
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Yu, & Pintrich, 1996), and intrinsic motivation (Elliot & Harackiewicz, 1996; Elliot & Church, 
1997). Perfonnance-approach goals also have been li nk ed to negative outcomes such as test 
anxiety (Elliot & McGregor, 1999; Middleton & Midgley, 1997), low self-efficacy (Skaalvik, 
1997), and higher avoidant help-seeking (Middleton & Midgley, 1997; Ryan & Pintrich, 1998). 
Performance-avoidance goals have been associated with negative outcomes, such as low 
absorption during task engagement (Elliot & Harackiewicz, 1996), an unwillingness to seek help 
with schoolwork (Middleton & Midgley, 1997), and reduced intrinsic motivation (Elliot & 
Church, 1997). Mastery-approach goals have been found to relate to positive outcomes, such as 
long-term retention of information (Elliot & McGregor, 1999), intrinsic motivation (Elliot & 
Church, 1997; Elliot & Harackiewicz, 1996), help-seeking (Ryan & Pintrich, 1998), and 
persistence (Pintrich, 2000). Finally, a mastery-avoid orientation has been found to correlate 
with negative outcomes such as low achievement (Crippen, Biesinger, Muis, & Orgill, 2009), 
disorganization and emotionality (Elliot & McGregor, 2001), and positively relate to outcomes 
such as fear of failure (Fryer & Elliot, 2007). 

Studies have also examined psychometric properties of various types of instruments used 
to measure achievement goals (Elliot & McGregor, 2001; Finney, Pieper, & Barron, 2004; 
Jagacinski & Duda, 2001; Midgley et ah, 1998). For example, Finney et al. (2004) examined the 
factor structure of the Achievement Goal Questionnaire (AGQ; Elliot & McGregor, 2001) in a 
general academic context. A sample of 2111 freshman undergraduate students completed the 
AGQ along with other various instruments. Results supported the four-factor structure that Elliot 
and McGregor (2001) hypothesized. Moreover, correlations between each of the four 
dimensions were low, and Cronbach alpha reliability coefficients were greater than .70, with the 
exception of the performance avoidance subscale (.54). 

Our review of the literature suggests the majority of studies have examined relations 
between achievement goals and other cognitive, affective, and achievement outcomes. 
Psychometric assessments of the various scales also have been conducted on specific samples 
(Elliot & McGregor, 2001), between groups to assess factorial invariance (Midgley et al., 1998), 
or across general or domain-specific contexts (Finney et al., 2004; Jagacinski & Duda, 2001). 
More recently, however, studies have explored the stability of goal orientations across contexts 
(Fryer & Elliot, 2007; Muis & Edwards, 2009; Senko & Harackiewicz, 2005; Winne, Muis, & 
Jamieson-Noel, 2005). As Pintrich (2000) stated, studies are needed to determine whether 
achievement goals are constant across contexts or whether they change as contexts change. 
Accordingly, theorists have suggested that achievement goals may not be as stable as theorists 
typically assume (Fryer & Elliot, 2007; Muis & Edwards, 2009; Senko & Harackiewicz, 2005; 
Winne et al., 2005). Results from the few studies that have examined goal stability found 
support for both stability and change (Fryer & Elliot, 2007; Muis & Edwards, 2009; Senko & 
Harackiewicz, 2005; Winne et al., 2005). 

For example, Senko and Harackiewicz (2005) examined whether individuals who 
received ongoing performance feedback on a series of tasks engaged in goal-switching (e.g., 
switching from one goal type to another) or goal intensification (e.g., to strengthen or reduce the 
pursuit of one goal). In the first study, tasks included four non-cumulative multiple-choice 
exams. For the second study in a laboratory setting, feedback was manipulated such that 
participants performed either “well below average” or “well above average” on a first 
mathematics task, and “well above average” on the second set of tasks. Results from Senko and 
Harackiewicz’s (2005) first study revealed that individuals’ goal orientations remained stable 
throughout the semester, although poor exam performance predicted a significant decrease in 
mastery-approach and performance-approach goals, and an increase in performance-avoidance 
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goals. In the second study, they found that negative feedback reduced individuals’ mastery- 
approach goals. 

In another study, Muis and Edwards (2009) examined the stability and change in 
students’ achievement goal orientations across varying tasks over the course of two 
undergraduate classes. For both classes, achievement goals were measured four times: prior to 
two assignments and two exams. Using four different complimentary techniques, results across 
both studies revealed evidence for stability and change across tasks. Moreover, the level of 
change that occurred across the different tasks was similar, which the authors argued suggested 
that task specificity was not a key factor in the amount of change that occurred over the course of 
the semester. Based on these results, they proposed that other factors, such as fear of failure, 
anxiety, or interest in a specific task may be more predictive of goal change than the task itself. 

We question, however, whether changes in students’ mean level of achievement goals 
were a function of true differences or a function of fluctuations in students’ interpretations of 
items across the task contexts. In particular, given that few studies have conducted psychometric 
assessments of the instruments researchers typically use to measure achievement goals, and 
given the new direction in this line of work, more research is needed to further establish 
construct validity and reliability. Particularly, it is imperative that factor analytic work is 
conducted to establish factorial invariance across contexts. Typically, factorial invariance is 
conducted across groups to assess whether group differences are meaningful and valid (i.e., ‘Can 
items be interpreted similarly across groups? [Chan, 2000]). Measurement equivalence needs to 
be established prior to examining mean differences across groups. If items are not equivalent, 
then differences between groups’ means cannot be meaningfully established—the construct may 
not be the same across groups. 

A parallel argument applies to differences in achievement goals across time or contexts. 
When students respond to questionnaire items after receiving performance feedback (Senko & 
Harackiewicz, 2005; Winne et ah, 2005) or are responding to individual items that focus 
specifically on a particular task (Fryer & Elliot, 2007; Muis & Edwards, 2009), context and 
content changes. Theoretically, because there may be cumulative effects of feedback or differing 
effects due to task specificity on item interpretation, it is not assured a priori that data on 
students’ goal orientations reflect the same constructs from time to time. 

We addressed these issues by investigating whether students’ goal orientation exhibited 
invariant factorial structure across various tasks after receiving feedback about those tasks. As 
Chan (2000) argued, this is essential before comparing observed data in exploring for differences 
across time and tasks. As well, we add to the empirical literature by further exploring the 
psychometric properties of one of the most widely used instruments, the Achievement Goal 
Questionnaire (AGQ; Elliot & McGregor, 2001). Specifically, we explored factorial invariance 
across four different task contexts, and examined construct- and discriminant-related evidence of 
validity and reliability of the items with a sample of 99 university undergraduate students 
enrolled in an introductory educational psychology course. Given the small sample size, 
precision of estimates may be affected and, as such, we suggest that results are interpreted with 
caution. 


Method 


Participants 

Participants were 99 students—13 males and 86 females—taking an educational 
psychology course. The mean age of students was 23.12 years (SD = 6.97), and the mean self- 
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reported GPA was 3.42 (SD = 2.96, N = 85). The sample was participating in a multi-faceted 
study and responded to other questionnaires about study tactics that are not relevant here. 

Measures 

Goal orientation. We used the 12-item Achievement Goal Questionnaire (AGQ; Elliot 
& McGregor, 2001) to assess students’ achievement goals for their educational psychology 
course. Students indicated their agreement with each statement on a scale of 1 (not at all true of 
me) to 7 (very true of me). The AGQ generates four theoretically orthogonal subscales of three 
items each: mastery-approach, mastery-avoidance, performance-approach, and performance- 
avoidance. Sample items are: “I desire to completely master the material presented in this class” 
(mastery-approach), “I worry that I may not learn all that I possibly could in this class” (mastery- 
avoid), “It is important for me to do better than other students” (performance-approach), and 
“My goal in this class is to avoid performing poorly” (performance-avoid). 

Procedure 

In this course, students wrote two five-page papers that required them to reason with 
course content and apply it to teaching. They also took two multiple-choice exams, a midterm 
covering the fist half of the course, and a final covering the second half of the course. In week 2 
of the semester, before participants were given any assignments or exams, they completed the 
AGQ. Each time an assignment or exam was handed back with feedback, they completed the 
AGQ again. Instructions and items for the AGQ were the same across all tasks. Students were 
asked to indicate how well each statement best described them. 

Results 

Descriptive Statistics and Discriminant Validity 

Using SPSS 19 (IBM), we first examined sub-scales of the AGQ for normality. Kline 
(1998) suggested using absolute cut-off values of 3.0 for skewness and 8.0 for kurtosis. All 
items on the AGQ were well within these ranges (ranging from -1.79 to 0 for skewness and from 
-1.18 to 4.88 for kurtosis). Means, standard deviations, and Cronbach alphas are reported in 
Table 1. To examine discriminant-related validity, we examined correlations between each of the 
theoretically defined goal orientation sub-scales. Correlations should be low to provide evidence 
of four distinct dimensions. As shown in Table 2, most correlations between sub-scales across 
the four time points were near zero. The exceptions were correlations between the two mastery 
orientation sub-scales and between the two avoidance sub-scales, which were moderate. We also 
examined correlations between and within each of the goal orientation sub-scales across time. As 
shown in Table 3, correlations within each goal orientation ranged from moderate to high, 
whereas correlations between each of the goal orientations were predominantly low, although 
some moderate correlations were found between the two mastery orientations. 



Descriptive Statistics and Cronbach as for Goal Orientations. 
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Table 2. 

Correlations Between Each Goal Orientation Across Tasks. 


Think Paper 1 

1 

2 

3 

MAP, 




mav 2 

.43** 



pap 3 

-.05 

.08 


pav 4 

-.01 

41** 

32** 

Midterm 

1 

2 

3 

MAP, 




mav 2 

.56** 



pap 3 

.03 

.02 


pav 4 

-.04 

.39* 

.12 

Think Paper 2 

1 

2 

3 

MAP, 




mav 2 

.50** 



pap 3 

-.07 

-.12 


pav 4 

-.10 

39 ** 

.13 


Note: MAP = mastery approach, MAY = mastery avoidance, PAP = performance approach, 
PAY = performance avoidance, * p < .05, and ** p < .01. 
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Table 3. 

Correlations Between Each Goal Orientation Across Time. 



Time 2 

Time 3 

Time 4 

Time 1 

1 

2 

3 

4 

1 

2 

3 

4 

1 

2 

3 

4 

MAPi 

.75 b 

.44 b 

.00 

-.10 

.66 b 

.40 b 

-.07 

-.12 

.66 b 

.54 b 

-.13 

-.03 

mav 2 

.37 b 

.72 b 

-.04 

.32 b 

.28 a 

.66 b 

-.13 

.35 b 

.37 b 

Os 

OO 

a* 

-.22 

.20 

pap 3 

-.05 

-.05 

.77 b 

.04 

-.12 

-.15 

.74 b 

.04 

-.03 

-.01 

.74 b 

-.01 

pav 4 

.01 

.31 b 

.20 

.67 b 

-.08 

.23 a 

.15 

.73 b 

-.07 

.15 

.16 

.60 b 

Time 2 




MAP 





OO 

80 

a* 

.53 b 

-.08 

-.04 

.92 b 

.57 b 

-.06 

-.03 

MAV 





.50 b 

oo 

CT 

.01 

.46 b 

.58 b 

.89 b 

-.18 

.36 b 

PAP 





-.01 

-.13 

.92 b 

.06 

-.04 

-.06 

.80 b 

-.01 

PAV 





-.12 

.35 b 

.05 

oo 

00 

o' 

-.06 

.34 b 

.03 

.86 b 

Time 3 




MAP 









.85 b 

.51 b 

-.12 

-.07 

MAV 









.44 b 

oo 

O 

cr 

-.16 

.36 a 

PAP 









-.05 

-.05 

.83 b 

.02 

PAV 









-.05 

.36 a 

.14 

.88 b 


Note: MAP = mastery approach, MAY = mastery avoidance, PAP = performance approach, 
PAY = performance avoidance , a p < .05, and h p < .01. 
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Evaluating Fit of the Factor Structure: Construct Validity 

EQS 6.0 (Multivariate Software Inc) was used to cross-validate Elliot and McGregor’s 
(2001) 2 x 2 model of goal orientation. We used data from the start of the course to test five 
competing models also tested by Elliot and McGregor (2001) and Finney et al. (2004) that might 
explain relationships among item responses. Model A was a four-factor model based on the 2 x 
2 framework for goal orientation. Model B was a three-factor model that included a performance 
approach factor, a performance-avoid factor, and an overall mastery factor. Model C was a 
three-factor model that included a mastery approach factor, a performance approach factor, and 
an overall avoidance factor. Model D was a two-factor model that included an overall mastery 
factor and an overall performance factor. Finally, Model E was a two-factor model that included 
an overall approach factor and an overall avoidance factor. Given that alternative models B 
through E are nested within the four-factor model (the hypothesized model, model A), a chi- 
square difference test (A^ 2 ) can be conducted between Model A and each respective model. If 
the Ax 2 is significant, the more complex model (Model A) fits significantly better than the 
alternative model. Accordingly, differences between chi-square statistics and the comparative fit 
index (CFI) were used to compare models. The CFI is particularly sensitive to misspecified 
factor pattern correlations and is a useful fit index when sample sizes are not large (Tabachnick 
& Fidell, 1996). A value of .95 or higher indicates good fit (Hoyle & Panter, 1995). 

Table 4 displays chi-square, Ayf, and CFI values for all five models. The chi-square and 
CFI values for model A were 83.63 {df = 48, N = 99, p < .01) and .95, respectively. This 
suggests a good fit to the data. For model B, the chi-square and CFI values were 127.63 ( df= 
51, p < .01) and .85, suggesting a moderate fit to the data. Chi-square and CFI values for model 
C were 127.54 ( df= 51,/? < .01) and .85, again, suggesting a moderate fit. Model D resulted in a 
chi-square of 176.02 {df = 53, p < .01) and a CFI of .75. Finally, model E resulted in a chi- 
square of 204.33 ( df = 53, p < .01) and a CFI of .70. Consistent with Elliot and McGregor’s 
(2001) and Finney et al.’s (2004) results, our data validate the four-factor model of goal 
orientation for mastery approach, mastery avoidance, performance approach, and performance 
avoidance. 
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Table 4. 

Fit Statistics for the Five Goal Orientation Models. 




Model 

r 

Ar 

A# 

CFI 

ACFI 

Four-Factor Model 

83.63 

(#=48) 



.93 


Overall Mastery 

127.63 

(#=51) 

44 

3 

.85 

-.08 

Overall Avoidance 

127.54 

(#=51) 

43.91 

3 

.85 

-.08 

Overall Mastery and 
Overall Performance 

176.02 
(#= 53) 

92.39 

5 

.75 

-.18 

Overall Approach and 
Overall Avoidance 

204.33 
(#= 53) 

120.70 

5 

.70 

-.23 


Invariance of the Factor Structure 

We then investigated whether students’ goal orientations exhibited invariant factorial 
structure across the different time points. Five steps are involved in testing for factorial 
invariance. In step one, the data are tested separately at each time for fit to the model (Byrne, 
1998), in our case, the 2x2 model of goal orientation. If this test is passed, four more tests are 
applied to establish that differences across time can be validly attributed to genuine differences 
rather than to differences in the architecture of the construct in different times, or to varying 
biases in measurements (Cheung & Rensvold, 2002). First, configural invariance is tested to 
examine whether the 2x2 architecture of the latent traits that gives rise to item responses does 
not vary across time. Second, metric invariance is tested by adding a constraint that loadings of 
the item responses on the latent constructs are invariant across time. Third, scalar invariance is 
examined to test whether the observed variables have the same degree of statistical bias—the 
distance of a mean from a fixed point—over time. This is accomplished by setting the influence 
of a latent trait to zero and computing predicted values of the observed variables at each time. 
There should be no differences (Hancock, 1997). Last, testing for measurement error invariance 
examines invariance of the factor variances and co-variances over time. Invariance of factor co- 
variances indicates that relationships among factors are similar across time, and invariance of 
factor variances indicates the ranges of scores on factors are similar over time. Because these 
two tests are statistically independent, order of assessment is irrelevant. 

Separate estimations of the fit of the data to Elliot and McGregor’s 2x2 model revealed 
adequate fit at each time. The chi-square and CFI values for the factor structure at time 1 (start 
of the course) were 83.63 ( df= 48, N= 99) and .95, respectively, as we reported previously. At 
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time 2, the chi-square and CFI values were 69.91 (#= 48, iV= 71) and .97, respectively. At time 

3, chi-square and CFI values were 84.04 (#= 48, N= 76) and .95, respectively. Finally, at time 

4, chi-square and CFI values were 74.26 ( df= 48, N = 53) and .95, respectively. 

Because our data adequately fit the 2x2 model of goal orientation at each time, we 
proceeded to test for invariance of measurement properties by adding successive constraints on 
parameters. The fit of the model after adding a constraint was compared to the model at the 
previous step in terms of differences in chi-square and CFI (Cheung & Rensvold, 2002). A 
ACFI value less than or equal to -.01 indicates that the null hypothesis of invariance should not 
be rejected (Cheung & Rensvold, 2002). If the fit of the more constrained model is statistically 
or practically worse than the less constrained model at the prior step, it would be concluded the 
parameters being constrained differ across time; that is, they are not invariant. In Table 5, we 
report statistics for these tests of configural invariance, metric invariance, scalar invariance, and 
measurement error invariance. All four levels of invariance were satisfied across the four points 
in time. 

Table 5. 

Tests of Invariance Across Contexts. 


Model 

r 

Ar 

A# 

CFI 

ACFI 

Configural Invariance 
(nothing held invariant) 

310.74 
(#= 192) 



.95 


Metric Invariance 
(coefficients held 
invariant) 

330.61 

(#=216) 

19.87 

24 

.95 

.00 

Scalar Invariance 
(coefficients and 
intercepts held invariant) 

359.91 
(#= 240) 

29.3 

24 

.95 

.00 

Measurement Error 
Invariance I 
(coefficients, intercepts 
and factor covariances 
held invariant) 

369.42 

(#=258) 

9.51 

18 

.95 

.00 

Measurement Error 
Invariance II 
(coefficients, intercepts, 
factor covariances and 
factor variances held 
invariant) 

388.53 

(#=270) 

19.11 

12 

.95 

.00 
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Reliability 

To assess the reliability of each of the four dimensions, Cronbach’s co-efficient alpha 
was calculated within each. With the exception of two of the estimates at the start of the course 
(for the mastery approach and performance-avoid subscales), reliabilities were all greater than 
.80, which indicated good internal consistency. Test-retest reliabilities were also computed 
across the four contexts. Values ranged from .60 to 74 (half of which were greater than .70), 
which indicates good test-retest reliability. Reliability statistics are presented in Tables 1 and 6 
(for test-retest). 

Table 6. 

Test-Retest Reliability Coefficients for Each Goal Orientation. 



^Time 1' Time 2 

max r Timel . 

Time 2 

^*Time l’Time 3 

max r Time 

l’Time 3 

^Time l’Time4 

rnaxr Timel . 

Time 4 

Mastery 

Approach 

.75 

.78 

.66 

.78 

.66 

.79 

Mastery- 

Avoid 

.72 

.86 

.66 

.89 

.68 

.87 

Performance 

Approach 

.77 

.93 

.74 

.92 

.74 

.93 

Performance- 

Avoid 

.67 

.75 

.73 

.76 

.60 

.77 


Note: max r timc represents the maximum correlation possible between time 1 and each subsequent time, 
computed by 7(a, *a n ). 


Discussion 

This study contributes to the literature on achievement goal theory by providing further 
evidence of construct and discriminant validity, reliability, and factorial invariance across tasks 
within a single classroom environment. For construct validity, results from our study 
demonstrate support for the four-factor model of achievement goals that Elliot and McGregor 
(2001) proposed. Compared to the four competing models, the 2x2 framework resulted in a 
much better fit than the trichotomous or dichotomous frameworks. The generalizability of the 
factor structure with our sample is an important addition to the few psychometric studies that 
have been done with achievement goal instruments in general. 

The low to moderate correlations between each of the achievement goals across the 
various contexts provides evidence of discriminant-ralted validity. Based on our results, we 
posit that each of the goal orientations represents a distinct construct. Like Elliot and McGregor 
(2001) and Finney et al. (2004), participants in our study endorsed each of the goal orientations 
across the various task contexts. Moreover, given that test-retest reliabilities were within an 
acceptable range, individuals’ levels of endorsements did not vary much over the different 
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contexts, nor did their interpretations of the items designed to measure the four dimensions. That 
is, our results unambiguously show that over time and across various contexts, students’ 
interpretations of the 12 items do not change. 

Substantively, results from the tests for factorial invariance indicate that items do not 
function differentially for any of the sub-scales across tasks. The range of each latent trait scale, 
which measures individuals’ levels of endorsement, is similar over time. Accordingly, full 
measurement equivalence at the item level is established across tasks and any direct mean 
comparisons to assess changes in levels of goal endorsement over time can be meaningfully 
made. Establishing factorial invariance across tasks and time is important given that research in 
this area is moving toward examining the stability of goal orientations over tasks, time, and as a 
function of feedback. In these situations, it is pertinent that research is conducted to examine 
item functioning over time. Our study addressed this gap in the literature. 

Limitations and Future Directions 

One limitation of our study is sample size. Although data from our sample fit the 
hypothesized model quite well, future research could use larger sample sizes. However, we note 
an important value that our study adds is the longitudinal and naturalistic nature of the design: 
students estimated their achievement goals over time and tasks within one classroom setting. 
Given that attrition is problematic with longitudinal designs, we recommend researchers begin 
with larger sample sizes to provide more power to detect slight variations in item interpretations. 
We also recommend more psychometric work on other instruments designed to measure 
students’ achievement goals like the Patterns of Adaptive Learning Scales (Midgley et ah, 2000). 
Studies are also needed to explore whether slight changes to items that are written for specific 
tasks, rather than for a specific course, influence item interpretation or construct validity. 
Psychometrically sound instruments that can be modified slightly without altering theoretical 
frameworks are needed to further explore the nature of achievement goals and how those goals 
influence important educational outcomes. 



246 


K. R. MU IS & P. WINNE 


References 

Ames, C. (1992). Classrooms: Goals, structures, and student motivation. Journal of Educational 
Psychology, 84, 261-271. 

Anderman, E. M., & Maehr, M. L. (1994). Motivation and schooling in the middle grades. 
Review of Educational Research, 64, 287-309. 

Byrne, B. M. (1998). Structural Equation Modeling with LISREL, PRELIS, and SIMPLIS. NJ: 
Lawrence Erlbaum Associates. 

Chan, D. (2000). Detection of differential item functioning on the Kirton Adaptation-Innovation 
Inventory using multiple-group mean and covariance structure analyses. Multivariate 
Behavior Research, 35, 169-199. 

Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing 
measurement invariance. Structural Equation Modeling, 9, 233-255. 

Crippen, K., Biesinger, K., Muis, K. R., & Orgill, M. K. (2009). The role of goal orientation and 
self-efficacy in learning from Web-based worked examples. Journal of Interactive 
Learning Research, 20, 385-403. 

Dweck, C. S., & Leggett, E. L. (1988). A social-cognitive approach to motivation and 
personality. Psychological Review, 95, 256-273. 

Elliot, A. J. (1999). Approach and avoidance motivation and achievement goals. Educational 
Psychologist, 34, 169-189. 

Elliot, A. J., & Church, M. A. (1997). A hierarchical model to approach and avoidance 

achievement motivation. Journal of Personality and Social Psychology, 72, 218-232. 

Elliot, A. J., & Dweck, C.S. (1988). Goals: An approach to motivation and achievement. Journal 
of Personality and School Psychology, 54, 5-12. 

Elliot, A. J., & Harackiewicz, M. (1996). Approach and avoidance achievement goals and 
intrinsic motivation: A mediational analysis. Journal of Personality and Social 
Psychology, 70,461-475. 

Elliot, A. J., & McGregor, H. (1999). Test anxiety and the hierarchical model of approach and 
avoidance achievement motivation. Journal of Personality and Social Psychology, 76, 
628-644. 

Elliot, A. J., & McGregor, H. (2001). A 2 X 2 achievement goal framework. Journal of 
Personality and Social Psychology, 80, 501-519. 

Finney, S. J., Pieper, S. L., & Barron, K. E. (2004). Examining the psychometric properties of 
the achievement goal questionnaire in a general academic context. Educational and 
Psychological Measurement, 64, 365-382. 

Fryer, J. W. & Elliott, A. J. (2007). Stability and change in achievement goals. Journal of 
Educational Psychology, 99, 700-714. 

Hancock, G. R. (1997). Structural equation modeling methods of hypothesis testing of latent 
variable means. Measurement and Evaluation in Counseling and Development, 30, 91- 
105. 



PSYCHOMETRIC ASSESSMENTS OF ACHIEVEMENT GOALS 


247 


Hoyle, R. H., & Panter, A. T. (1995). Writing about structural equation models. In R. H. Hoyle 
(Ed.), Structual equation modeling: Concepts, issues and applications (pp. 159-176). 
Thousand Oaks, CA: Sage. 

Jagacinski, C. M., & Duda, J. L. (2001). A comparative analysis of contemporary achievement 
goal measures. Educational and Psychological Measurement, 61, 1013-1039. 

Kline, R. (1998). Principles and practice of structural equation modeling. New York: Guilford 
Press. 

Maehr, M. L., & Pintrich, P. R. (1991). Advances in motivation and achievement: Goals and 
self-regulatory processes. Greenwich, CT. JAI Press. 

Meece, J. L. (1991). The classroom context and students’ motivational goals. In M. L. Maehr & 
P. R. Pintrich (Eds.), Advances in motivation and achievement: Goals and self-regulatory 
processes (pp. 261-285). Greenwhich, CT: JAI Press. 

Middleton, M. J., & Midgley, C. (1997). Avoiding the demonstration of lack of ability: An 
unexpected aspect of goal theory. Journal of Educational Psychology, 89, 710-718. 

Midgley, C., Kaplan, A., & Middleton, M. (2001). Performance-approach goals: Good for what, 
for whom, under what circumstances, and at what cost? Journal of Educational 
Psychology, 93, 77-86. 

Midgley, C., Kaplan, A., Middleton, M., Maehr, M. L., Urdan, T., Anderman, L. H., Anderman, 
E., & Roeser, R. (1998). The development and validation of scales assessing students’ 
achievement goal orientations. Contemporary Educational Psychology, 23, 113-131. 

Midgley, C., Maehr, M. L., Hruda, L. Z., Andennan, E., Anderman, L., Freeman, K. E., et al. 
(2000). Manual for the Patterns of Adaptive Learning Scales (PALS). Retrieved from 
www.unmich-edu/~pals/pals . 

Muis, K. R., & Edwards, O. V. (2009). Examining the stability of achievement goal orientations. 
Contemporary Educational Psychology, 34, 265-277. 

Pintrich, P. R. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, 

P. R. Pintrich & M. Zeidner (Eds), Handbook of Self-regulation. San Diego, CA: 
Academic Press. 

Ryan, A. M., & Pintrich, P. R. (1998). Achievement and social motivational influences on help 
seeking in the classroom. In S. A. Karabenick (Ed.), Strategic help seeking: Implications 
for learning and teaching (pp. 117-139). Mahwah, NJ: Erlbaum. 

Senko, C., & Harackiewicz, J. M. (2005). Regulation of achievement goals: The role of 
competence feedback. Journal of Educational Psychology, 97, 320-336. 

Skaalvik, E. M. (1997). Self-enhancing and self-defeating ego orientation: Relations with task 
and avoidance orientation, achievement, self-perceptions, and anxiety. Journal of 
Educational Psychology, 89, 71-81. 

Tabachnick, B. G., & Fidell, L. S. (2001). Using multivariate statistics (4th ed.). New York: 
Harper Collins. 




248 


K. R. MU IS & P. WINNE 


Wolters, C., Yu, S., & Pintrich, P. R. (1996). The relation between goal orientation and students’ 
motivational beliefs and self-regulated learning. Learning and Individual Differences, 8, 
211-238. 

Winne, P. H., Muis, K. R., & Jamieson-Noel, D. J. (2005). Relationships among achievement 

goal orientation, calibration bias and performance in response to successive feedback in 
an undergraduate course. Paper presented at the annual American Educational Research 
Association. 



